Deep neural networks for automated detection of marine mammal species

Authors thank the Bureau of Ocean Energy Management for the funding of MARU deployments, Excelerate Energy Inc. for the funding of Autobuoy deployment, and Michael J. Weise of the US Office of Naval Research for support (N000141712867).Deep neural networks have advanced the field of detection and classification and allowed for effective identification of signals in challenging data sets. Numerous time-critical conservation needs may benefit from these methods. We developed and empirically studied a variety of deep neural networks to detect the vocalizations of endangered North Atlantic right whales (Eubalaena glacialis). We compared the performance of these deep architectures to that of traditional detection algorithms for the primary vocalization produced by this species, the upcall. We show that deep-learning architectures are capable of producing false-positive rates that are orders of magnitude lower than alternative algorithms while substantially increasing the ability to detect calls. We demonstrate that a deep neural network trained with recordings from a single geographic region recorded over a span of days is capable of generalizing well to data from multiple years and across the species’ range, and that the low false positives make the output of the algorithm amenable to quality control for verification. The deep neural networks we developed are relatively easy to implement with existing software, and may provide new insights applicable to the conservation of endangered species.Publisher PDFPeer reviewe

Singing fin whale swimming behavior in the Central North Pacific

This research was supported by Commander, U.S. Pacific Fleet (Code N465JR, Award Number N0007020WR0EP8F), the Office of Naval Research (Code 322, Award Number N0001421WX00156), and tool development necessary for this analysis was supported by the U.S. Navy's Living Marine Resources Program (Award Number N0002520WR0141R).Male fin whales sing using 20 Hz pulses produced in regular patterns of inter-note intervals, but little is known about fin whale swimming behavior while they are singing. Even less is known about fin whales in Hawaiian waters because they have rarely been sighted during surveys and passive acoustic monitoring has been limited to sparse hydrophone systems that do not have localization capabilities. We hypothesized that fin whale kinematics may be related to their singing behavior, or external variables such as time and sea state. To investigate this hypothesis, we analyzed 115 tracks containing 50,034 unique notes generated from passive acoustic recordings on an array of 14 hydrophones from 2011 to 2017 at the U.S. Navy Pacific Missile Range Facility off Kauai, Hawaii. Fin whales swam at an average speed of 1.1 m/s over relatively direct paths. We incorporated the whales' speed and turning angle into hidden Markov models to identify different behavioral states based on the whales' movements. We found that fin whale kinematic behavioral state was related to the vocalization rate (also known as cue rate) and time of day. When cue rate was higher, fin whales were more likely to swim slower and turn more than when cue rate was lower. During the night, fin whales were also more likely to swim slower and turn more than during the day. In addition, we examined whether the presence of singing fin whales was related to time and sea state using generalized additive models. Fin whale track presence was affected by day of the year and song season, and possibly also wind speed and wave height. Although the track kinematics from the fin whale tracks presented here are limited to a subset of whales that are acoustically active, they provide some of the only detailed movements of fin whales in the region and can be compared against fin whale swim speeds in other regions. Understanding how fin whale swimming behavior varies based on their vocalization patterns, time, and environmental factors will help us to contextualize potential changes in whale behavior during Navy training and testing on the range.Publisher PDFPeer reviewe

Improve automatic detection of animal call sequences with temporal context

Funding: This work was supported by the US Office of Naval Research (grant no. N00014-17-1-2867).Many animals rely on long-form communication, in the form of songs, for vital functions such as mate attraction and territorial defence. We explored the prospect of improving automatic recognition performance by using the temporal context inherent in song. The ability to accurately detect sequences of calls has implications for conservation and biological studies. We show that the performance of a convolutional neural network (CNN), designed to detect song notes (calls) in short-duration audio segments, can be improved by combining it with a recurrent network designed to process sequences of learned representations from the CNN on a longer time scale. The combined system of independently trained CNN and long short-term memory (LSTM) network models exploits the temporal patterns between song notes. We demonstrate the technique using recordings of fin whale (Balaenoptera physalus) songs, which comprise patterned sequences of characteristic notes. We evaluated several variants of the CNN + LSTM network. Relative to the baseline CNN model, the CNN + LSTM models reduced performance variance, offering a 9-17% increase in area under the precision-recall curve and a 9-18% increase in peak F1-scores. These results show that the inclusion of temporal information may offer a valuable pathway for improving the automatic recognition and transcription of wildlife recordings.Publisher PDFPeer reviewe

Changes in the movement and calling behavior of minke whales (Balaenoptera acutorostrata) in response to navy training

This research was funded by the U.S. Office of Naval Research under grant number N000141612859. The passive acoustic data were recorded under support by COMPACFLT for the Navy Marine Species Monitoring Program. The call association tracking algorithm was developed under a separate U.S. Office of Naval Research project (2011–2015 Advanced Detection, Classification and Localization, grant number: N0001414IP20037).Many marine mammals rely on sound for foraging, maintaining group cohesion, navigation, finding mates, and avoiding predators. These behaviors are potentially disrupted by anthropogenic noise. Behavioral responses to sonar have been observed in a number of baleen whale species but relatively little is known about the responses of minke whales (Balaenoptera acutorostrata). Previous analyses demonstrated a spatial redistribution of localizations derived from passive acoustic detections in response to sonar activity, but the lack of a mechanism for associating localizations prevented discriminating between movement and cessation of calling as possible explanations for this redistribution. Here we extend previous analyses by including an association mechanism, allowing us to differentiate between movement responses and calling responses, and to provide direct evidence of horizontal avoidance responses by individual minke whales to sonar during U.S. Navy training activities. We fitted hidden Markov models to 627 tracks that were reconstructed from 3 years of minke whale (B. acutorostrata) vocalizations recorded before, during, and after naval training events at the U.S. Navy's Pacific Missile Range Facility, Kauai, Hawaii. The fitted models were used to identify different movement behaviors and to investigate the effect of sonar activity on these behaviors. Movement was faster and more directed during sonar exposure than in baseline phases. The mean direction of movement differed during sonar exposure, and was consistent with movement away from sonar-producing ships. Animals were also more likely to cease calling during sonar. There was substantial individual variation in response. Our findings add large-sample support to previous demonstrations of horizontal avoidance responses by individual minke whales to sonar in controlled exposure experiments, and demonstrate the complex nature of behavioral responses to sonar activity: some, but not all, whales exhibited behavioral changes, which took the form of horizontal avoidance or ceasing to call.Publisher PDFPeer reviewe

Learning deep models from synthetic data for extracting dolphin whistle contours

We present a learning-based method for extracting whistles of toothed whales (Odontoceti) in hydrophone recordings. Our method represents audio signals as time-frequency spectrograms and decomposes each spectrogram into a set of time-frequency patches. A deep neural network learns archetypical patterns (e.g., crossings, frequency modulated sweeps) from the spectrogram patches and predicts time-frequency peaks that are associated with whistles. We also developed a comprehensive method to synthesize training samples from background environments and train the network with minimal human annotation effort. We applied the proposed learn-from-synthesis method to a subset of the public Detection, Classification, Localization, and Density Estimation (DCLDE) 2011 workshop data to extract whistle confidence maps, which we then processed with an existing contour extractor to produce whistle annotations. The F1-score of our best synthesis method was 0.158 greater than our baseline whistle extraction algorithm (~25% improvement) when applied to common dolphin (Delphinus spp.) and bottlenose dolphin (Tursiops truncatus) whistles.Postprin

North Pacific minke whales call rapidly when calling conspecifics are nearby

This research was supported by Commander, U.S. Pacific Fleet (Code N465JR, Award Number N0007020WR0EP8F) and tool development utilized for this analysis was supported by the U.S. Navy’s Living Marine Resources Program (Award Number N0002520WR0141R).North Pacific minke whale (Balaenoptera acutorostrata) boing calls are commonly detected in Hawaiian waters. When producing boing vocalizations, minke whales seem to be in one of two calling behavioral states. Most often minke whales produce boings with inter-call intervals of several minutes, but sometimes minke whales call rapidly with inter-call intervals of less than a minute. Since minke whales are difficult to detect visually, cue-rate-based density estimation using passive acoustic monitoring has been proposed. However, the variables that influence cue rate or calling rate are poorly understood in most whales, including minke whales. We collected passive acoustic recordings from 47 bottom-mounted hydrophones at the Pacific Missile Range Facility’s instrumented range off the coast of Kauaʻi, Hawaiʻi to test the hypothesis that minke whales call more rapidly when closer in proximity to other calling conspecifics. A total of 599 days of data were recorded between August 2012 and July 2017 and were automatically post-processed to detect, classify, and localize calls. Localized calls were grouped into tracks and manually validated, resulting in 509 individual tracks composed of 36,033 calls within a 16 x 39 km focal study area. Tracked minke whales exhibited a strong bimodal call rate with means of one call every 6.85 min (σ= 2.54 min) and 0.63 min (σ= 0.36 min). We ran hidden Markov models to quantify the relationship between call rate and the distance to the nearest calling conspecific. Overall, the probability of the higher call rate occurring increased as the distance to the nearest conspecific decreased, and the probability of the lower call rate occurring increased as the distance to the nearest conspecific increased. We also examined individual track data and found that minke whales may also exhibit other responses (i.e. increased speed, changes in heading, and cessation of calling) when calling conspecifics are nearby. These findings provide new information about minke whale calling behavior in what is likely a breeding area.Publisher PDFPeer reviewe

The Lombard effect in singing humpback whales : source levels increase as ambient ocean noise levels increase

Funding: Office of Naval Research (Code 322, Marine Mammals and Biology), Commander, U.S. Pacific Fleet (Code N465JR), and the Naval Facilities Engineering Command Living Marine Resources Program.Many animals increase the intensity of their vocalizations in increased noise. This response is known as the Lombard effect. While some previous studies about cetaceans report a 1 dB increase in the source level (SL) for every dB increase in the background noise level (NL), more recent data have not supported this compensation ability. The purpose of this study was to calculate the SLs of humpback whale song units recorded off Hawaii and test for a relationship between these SLs and background NLs. Opportunistic recordings during 2012-2017 were used to detect and track 524 humpback whale encounters comprised of 83 974 units on the U.S. Navy's Pacific Missile Range Facility hydrophones. Received levels were added to their estimated transmission losses to calculate SLs. Humpback whale song units had a median SL of 173 dB re 1 μ Pa at 1 m, and SLs increased by 0.53 dB/1 dB increase in background NLs. These changes occurred in real time on hourly and daily time scales. Increases in ambient noise could reduce male humpback whale communication space in the important breeding area off Hawaii. Since these vocalization changes may be dependent on location or behavioral state, more work is needed at other locations and with other species.Publisher PDFPeer reviewe